1. Field
The present invention relates to image processing systems and specifically to a system for calculating functions of intensity and summing the functions over an image frame.
2. Related Art
Image processing often requires performing a specified function over every pixel x of a given image and then summing results of the function for the pixels (i.e., Σƒ[x]). Typical examples of functions ƒ[x] required in processing of pixels in an image are sum of the intensity squared or the sum of intensity gradients. Such calculations are computationally intensive since conventional circuitry typically reads through the entire image for each summation of every desired function. Sometimes an area of an image known as a “window” is specified and used to obtain function values of a specified function ƒ[x] over the window. Accumulation and storage of multiple function values is typically performed with multiple passes through the image window. During each pass, intensity data is read from the pixels, and each particular function is calculated during each a unique pass through the window. Results of the functions are accumulated in local memory cells for each window.
Reference is now made to FIG. 1 (conventional art) and FIG. 2 (conventional art), which illustrate a driver assistance system 16 including a camera or image sensor 12 mounted in a vehicle 18 imaging a field of view in the forward or rearward direction. Image sensor 12 typically captures images in real time in a time series of image frames 15. An image processor 14 is used to process image frames 15 to perform one of a number of driver assistance systems.
During the last few years camera based driver assistance systems (DAS) 16 have been entering the market; including lane departure warning (LDW), Automatic High-beam Control (AHC), traffic sign recognition (TSR) and forward collision warning (FCW). Lane departure warning (LDW) systems are designed to give a warning in the case of unintentional lane departure. The warning is given when the vehicle crosses or is about to cross the lane marker. Driver intention is determined based on use of turn signals, change in steering wheel angle, vehicle speed and brake activation. There are various LDW systems available. One algorithm for lane departure warning (LDW) used by the assignee (Mobileye Technologies Ltd., Nicosia, Cyprus, hereinafter “Mobileye”) of the present application is predictive in that it computes time to lane crossing (TLC) based on change in wheel-to-lane distance and warns when the time-to-lane crossing (TLC) is below a certain threshold. Typically, the lane markers are detected in the camera image and then, given the known camera geometry and camera location relative to the vehicle, the position of the vehicle relative to the lane is computed. The lane markers detected in the camera image are then collected over time, for instance using a Kalman filter.
The core technology behind forward collision warning (FCW) systems and headway distance monitoring is vehicle detection. Assume that reliable detection of vehicles in a single image a typical forward collision warning (FCW) system requires that a vehicle image be 13 pixels wide, then for a car of width 1.6 m, a typical camera (640×480 resolution and 40 deg FOV) gives initial detection at 115 m and multi-frame approval at 100 m. A narrower horizontal field of view (FOV) for the camera gives a greater detection range however; the narrower horizontal field of view (FOV) will reduce the ability to detect passing and cutting-in vehicles. A horizontal field of view (FOV) of around 40 degrees was found by Mobileye to be almost optimal (in road tests conducted with a camera) given the image sensor resolution and dimensions. A key component of a typical forward collision warning (FCW) algorithm is the estimation of distance from a single camera and the estimation of scale change from the time-to-contact/collision (TTC) as disclosed for example in U.S. Pat. No. 7,113,867.
Traffic sign recognition (TSR) modules are designed typically to detect speed limit signs and end-of-speed limit signs on highways, country roads and urban settings. Partially occluded, slightly twisted and rotated traffic signs are preferably detected. Systems implementing traffic sign recognition (TSR) may or should ignore the following signs: signs on truck/buses, exit road numbers, minimum speed signs, and embedded signs. A traffic sign recognition (TSR) module which focuses on speed limit signs does not have a specific detection range requirement because speed limit signs only need to be detected before they leave the image. An example of a difficult traffic sign to detect is a 0.8 meter diameter traffic sign on the side of the road when the vehicle is driving in the center lane of a three lane highway. Further details of a TSR system is disclosed by the present assignee in U.S. Patent Publication No. 2008/0137908.
Given that forward collision warning (FCW), traffic sign recognition (TSR) and lane departure warning (LDW) already require a high resolution monochrome sensor, a new automatic high-beam control (AHC) algorithm was developed for use with high resolution monochrome sensors as disclosed in U.S. Pat. No. 7,566,851. A number of different pattern recognition techniques are used with higher resolution monochrome imaging sensors to identify light sources instead of relying on color information. The automatic high-beam control (AHC) algorithm includes the following features: Detect bright spots in the sub-sampled long exposure image and then perform clustering and classification in the full resolution image, classify spots based on brightness, edge shape, internal texture, get further brightness information from the short exposure frames and classify obvious oncoming headlights based on size and brightness, track spots over time and compute change in size and brightness, pair up matching spots based on similarity of shape, brightness and motion, classify pairs as oncoming or taillights based on distance, brightness and color, and estimate distance and where unmatched spots might be motorcycles taillights.
Thus, there is a need for and it would be advantageous to have a multifunction summing machine to enable “bundling” of multiple driver assistance systems (e.g. automatic high-beam control (AHC) and traffic sign recognition (TSR), lane departure warning (LDW), forward collision warning (FCW)) on a single hardware platform, e.g. camera and processor. Bundling provides cost reduction and may allow more driver assistance functions to be added to the vehicle without increasing the space required beyond the windshield of the vehicle.
The terms “frame” or “image frame” as used herein is one of a sequence of pictures as output from a camera, typically a video camera. The terms “window” or “image window” as used herein is a portion of an image frame or the same portion over multiple image frames.